The present invention relates to vision systems and more specifically to a system and method for developing vision inspection programs for use by vision inspections systems. The resulting programs, when deployed, can be used to inspect parts and/or provide position information for guiding automated manipulations of such parts and/or machinery with respect to such parts.
Machine vision systems play an important role in automated systems. Cameras are used to obtain images of articles, and image processing is performed to identify features of the articles. Further image processing may be performed to identify an article's position, measure the article's dimensions, and/or to check for article defects. Image processing results may then be used to aid in the control of automated system components such as factory equipment, including, for example, an industrial controller, a robotic arm or a positioning table. Such machine vision systems may aid in the inspection, assembly, and/or handling of various types of articles, parts, and devices, including automotive parts (e.g., fuses, gaskets, and spark plugs), electrical components (e.g., connector pins, keyboards, LED, LCD, VFD displays), medical and pharmaceutical products (e.g., disposable test kits, syringes, needles, and date-lot codes), and consumer products (e.g., razor blades and floppy disks).
Whenever a new article, part, or device is inspected by a machine vision system, the vision processing component of the machine vision system will usually be modified to provide a new inspection program for the new item. For example, a system for automatically assembling (mounting) surface-mounted devices (SMDs) may run a vision inspection program for determining various characteristics (e.g., dimensions between edges, hole diameters, relative orientation to a surface to which the SMD is to be mounted, etc.) of each SMD as part of the mounting process. A different vision inspection program may be required to determine if a part that forms the surface to which an SMD is to be mounted is present, the part's orientation, part characteristics, etc. Where a first vision inspection program is used with a first SMD having a first characteristic set and a second vision inspection program is used with a second SMD having a second characteristic set, the first and second visions inspection programs may be different to identify the first and second different characteristic sets, respectively. Thus, for each new application that is not identical to an old application, a new inspection program typically has to be developed.
To enable custom vision inspection program design, sophisticated vision system manufacturers/providers typically bundle vision inspection program specifying software with system hardware where the software is used by a vision inspection developer (e.g., a customer) to select vision inspection system features, options or tools, to set inspection system variables and values, to select image features to be analyzed, to formulate useful vision inspection functions and expressions, etc. Here, the bundled software often includes a huge number (e.g., one hundred) of software based vision inspection tools that can be selected and for which variables can be set to build virtually an endless number of custom vision inspection programs to meet requirements of literally an endless list of applications. The software based inspection tools are typically specifiable via an interface presented on a computer display screen.
One interface for defining machine vision inspection programs is referred to generally as a scripting or programming interface. A scripting interface is one where an inspection system developer actually enters lines of program instructions that, when run by an inspection system, cause the system to perform a desired inspection process. In general a scripting interface is referred to as a full featured interface because the interface enables a user to specify or develop all inspection system features that are supported by the inspection system. While scripting interfaces have been useful in many applications, such interfaces are particularly difficult to use and require highly skilled developers trained in various programming languages.
One other useful inspection system specifying interface is referred to as a spreadsheet interface. U.S. Pat. No. 7,107,519 which issued on Sep. 12, 2006 and which is titled “Spread-Sheet Based User Interface Creation” and which is assigned to Cognex Corporation, describes an exemplary spreadsheet interface for programming vision inspection systems. Similarly, U.S. patent application Ser. No. 09/635,365 filed “Machine Vision Sensor” which was filed on Aug. 9, 2000 teaches various aspects of a spreadsheet interface. Both the patent and application referenced here are incorporated by reference in their entirety. As the label implies, the spreadsheet interface presents a spreadsheet image via a display screen that includes rows and columns of cells. Cell content is specified by a developer during an options specifying process and may include functions, mathematical expressions, data labels, or data to be processed according to an inspection program. In addition input and/or output functions can be placed in cells where input functions read information from hardware devices and output functions interact with specialized hardware devices. Together, the cell expressions in a spreadsheet define the inspection program. The spreadsheet interface, like the scripting interface, is a full featured interface because it can be used to specify or develop all inspection system features that are supported by the inspection system.
In at least some known spreadsheet interface systems, an inspection program developer can select predefined inspection objects or tools from a list of possible tools to be added to an inspection program. Here, each tool is often associated with a multi-step function set that is performed during an inspection process. When an inspection tool is selected, a set of spreadsheet cells is filled in with functions, expressions, labels and data that together define the inspection tool process. In some cases a dialog box opens up where a selected tool requires the developer to specify some value or values, input(s) or output(s).
In addition to placing functions and expressions in cells when predefined inspection tools are selected, a developer can develop custom tools, expressions and the like in any cell where other cells are referenced. Thus, a developer can use the spread sheet to develop custom inspection tools.
While the spreadsheet view is useful to develop inspection programs, spreadsheet views are not completely intuitive and the cell functions, expressions and data have a complicated format and a relatively messy appearance such that it takes some time to become proficient at programming with a spreadsheet interface and the end product is somewhat intimidating to view and work with during and after the programming process.
Yet one other type of particularly useful interface is generally referred to as a point-and-click (P&C) interface. As the label implies, a P&C interface allows a developer to select inspection options, set operating parameters, etc., using a point-and-click input device such as a mouse that controls an on screen cursor or the like. Here, the software can operate to guide a developer along the process of developing an inspection program. When some point-and-click options are selected, parameter fields or boxes may be opened to allow a developer to specify or selecting various inspection tool operating parameters. As inspection tools and operating parameters are selected and specified, the selections are used behind the scenes by a compiler to generate scripted programming code, functions and expressions which define the overall system inspection program.
One problem with some inspection programming systems is that the number of inspection tools supported by the systems and that the systems are capable of supporting can be huge (e.g., hundreds) such that, if presented in their entirety, the tool options could cause confusion for developers and could exacerbate the task of becoming a proficient inspection program developer.
Here, it has been recognized that, while a complete system may include several hundred inspection tools to choose from and may be able to support a virtually endless number of different inspection tools, (1) a relatively small subset of available/possible inspection tools are typically common among most (e.g., a large percentage of) vision inspection programs and (2) that the small subset of common inspection tools often represent a large percentage of inspection tools needed to instantiate any given inspection program. Thus, for instance, assume that an inspection specifying program supports 200 possible inspection tools and could support a virtually endless set of inspection tools (i.e., if the tools were specified). Also assume that two different programs are developed to inspect two different parts. Here, where the first inspection program for a first part includes fifteen different tools (and may include multiple instances of each of the fifteen different tools), it is very likely that the second inspection program will include a large number of the 15 tools (e.g., 13-14) included in the first program. Where the second program includes a total of 15 tools and 13 of the 15 tools are common with the first program, both the first and second programs could be developed using an inspection specifying program that includes only 17 inspection tools (i.e., the 15 tools for the first program and the additional 2 uncommon tools required to develop the second program) instead of a program including all 200 inspection options or a program offering an endless number of inspection tool options.
Similarly, where 20 different inspection programs are to be developed, it may be that all 20 programs could be developed using an inspection specifying program that includes only 25 different inspection tools instead of a program including 200 or more tools. Thus, by providing a point-and-click type specifying tool that includes a subset of available/possible tools, developer confusion is minimized and ease of use is increased appreciably.
As another refinement, inspection specifying programs can be developed for specific industries where the subset of possible inspection options that is made available to developers is specifically tailored to the specific industry. For instance, industries that need to determine liquid levels in clear containers may be provided with a different subset of available inspection options than industries that need to measure dimensions between parallel edges. In this case, in the liquid level measuring industry 10 inspection options may be usable to define 99% of all inspection programs while 40 options are required in the dimension measuring industry to define 99% of all programs.
While P&C interfaces that present only a subset of available inspection tools work well in some cases, at times a developer will desire or need inspection tools that, while available or at least possible using the full suite of development tools, are not presented via the P&C interface because of decisions made by the interface designer. Thus, for instance, in a case where 25 inspection options are presented to a developer via a point-and-click type interface, at times the developer may want to use eight other available or possible options in specific applications.
Here one solution has been to provide a programming interface that includes both point-and-click and scripting/programming capabilities where a P&C interface presents a subset of available inspection tools and a scripting interface can be used to specify the balance of required inspection tools. Here, when the point-and-click interface is accessed, the point-and-click interface operates as a front end to the scripting interface so that, as inspection tools are selected using the point-and-click interface, program segments are filled in with functions, expressions and data consistent with the point-and-click interface selections. When a developer needs a tool that is not presented by the point-and-click interface, the developer switches views to the scripting view and specifies other tools or defines other required functions.
While dual P&C and scripting interface systems can be used to successfully program vision processes, currently known dual interface systems have several important shortcomings. First, in known dual P&C/scripting interface systems, after a user moves from the P&C view to the scripting view and adds a new vision processing tool or changes a tool parameter using the scripting view, if the user moves back to the P&C view, there is no representation of the new vision processing tool or changed parameter in the P&C view. Inability to see P&C representations of scripted tools can generally cause confusion for inexperienced developers during an inspection system specifying process.
Second, known systems do not have a way for a developer to create new P&C interface tools that the developer would like to use routinely during subsequent development activities. For instance, a developer may routinely want to employ an inspection tool that performs a sequence of fifteen inspection functions to yield one or more useful results and that tool may not be selectable via the P&C interface so that, each time the developer wants to use the tool, the developer has to go to the scripting interface and recreate the fifteen function sequence. Clearly the inability to specify the fifteen function sequence once and represent that function for selection via the P&C interface thereafter exacerbates the task of developing inspection systems considerably.
Third, developers cannot change existing P&C tools to hide existing properties, change behaviors of existing tools, change default properties and/or behaviors of P&C tools such that all future instances will use the new defaults or tie behaviors of two or more tool instances together. These short comings mean that tool customization has to be done using scripting alone and result in a system where routine customization has to be performed repetitively which exacerbates the development process.
Thus, it would be advantageous to have an inspection program specifying system that allows a developer to move freely between scripting and P&C interfaces where at least some scripted customization is represented in the P&C interface and where P&C tool modification and new P&C tools specified in script can be saved to persist for use thereafter.